Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Docs] upgrade/chain halt recovery #837

Merged
merged 43 commits into from
Jan 24, 2025
Merged

[Docs] upgrade/chain halt recovery #837

merged 43 commits into from
Jan 24, 2025

Conversation

okdas
Copy link
Member

@okdas okdas commented Sep 24, 2024

Summary

Performed the first upgrade on the Alpha TestNet. Add some documentation changes to prevent some issues in the future.

Issue

N/A

Type of change

Select one or more from the following:

Testing

  • Documentation: make docusaurus_start; only needed if you make doc changes
  • Unit Tests: make go_develop_and_test
  • LocalNet E2E Tests: make test_e2e
  • DevNet E2E Tests: Add the devnet-test-e2e label to the PR.

Sanity Checklist

  • I have tested my changes using the available tooling
  • I have commented my code
  • I have performed a self-review of my own code; both comments & source code
  • I create and reference any new tickets, if applicable
  • I have left TODOs throughout the codebase, if applicable

@okdas okdas self-assigned this Sep 24, 2024
@okdas okdas added the documentation Improvements or additions to documentation label Sep 24, 2024
@okdas okdas added this to the Shannon Beta TestNet Launch milestone Sep 24, 2024
@okdas okdas requested a review from Olshansk September 24, 2024 23:50
@okdas okdas changed the title [Upgrades] Document learnings from the past upgrade [Upgrades] Document learnings from the last upgrade Sep 25, 2024
@okdas
Copy link
Member Author

okdas commented Sep 26, 2024

Items to add:

  • Configure cosmovisor to create backups before upgrade
  • Have more bump validators to 5.
  • Instructions on how to test the upgrade locally. Thinking a script w/o LocalNet.

@Olshansk
Copy link
Member

@okdas Let's add the details here as well: https://x.com/olshansky/status/1846211059989778741

@okdas okdas marked this pull request as ready for review October 17, 2024 01:18
@okdas
Copy link
Member Author

okdas commented Oct 17, 2024

@Olshansk can I get a review on this PR, please?

I think I want to investigate/add information about the new Cosmovisor feature mentioned in cosmos-sdk slack, but I'll look into that later. :)

@okdas okdas changed the title [Upgrades] Document learnings from the last upgrade [Upgrades] Document upgrade/chain halt recovery Oct 17, 2024
@okdas okdas changed the title [Upgrades] Document upgrade/chain halt recovery [Docs] upgrade/chain halt recovery Oct 17, 2024
@github-actions GitHub Actions
Copy link

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

You may need to run make trigger_ci to submit an empty commit that'll trigger the tests.

GCP workloads (requires changing the namespace to 837)
Grafana network dashboard for devnet-issue-837

@github-actions github-actions bot added devnet push-image CI related - pushes images to ghcr.io labels Oct 17, 2024
bryanchriswhite
bryanchriswhite previously requested changes Nov 20, 2024
View reviewed changes
Copy link
Contributor

@bryanchriswhite bryanchriswhite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great docs @okdas! 🙌 😍

app/upgrades/historical.go Outdated Show resolved Hide resolved
docusaurus/docs/develop/developer_guide/chain_halt_troubleshooting.md Outdated Show resolved Hide resolved
docusaurus/docs/develop/developer_guide/chain_halt_troubleshooting.md Show resolved Hide resolved
docusaurus/docs/develop/developer_guide/recovery_from_chain_halt.md Outdated Show resolved Hide resolved
docusaurus/docs/develop/developer_guide/recovery_from_chain_halt.md Outdated Show resolved Hide resolved
docusaurus/docs/protocol/upgrades/contigency_plans.md Outdated Show resolved Hide resolved
docusaurus/docs/protocol/upgrades/upgrade_procedure.md Show resolved Hide resolved
docusaurus/docs/protocol/upgrades/upgrade_procedure.md Outdated Show resolved Hide resolved
docusaurus/docs/protocol/upgrades/upgrade_procedure.md Show resolved Hide resolved
docusaurus/docs/protocol/upgrades/upgrade_procedure.md Show resolved Hide resolved
Olshansk and others added 11 commits November 20, 2024 11:38
@Olshansk
WIP review
7113cde
@Olshansk
Merge with main
739a6d3
@Olshansk
Merge branch 'main' into dk-upgrade-learnings
d8c448c
@Olshansk @bryanchriswhite
Update docusaurus/docs/develop/developer_guide/recovery_from_chain_ha…
ec4314c 
…lt.md

Co-authored-by: Bryan White <[email protected]>
@Olshansk @bryanchriswhite
Update docusaurus/docs/protocol/upgrades/contigency_plans.md
9886220 
Co-authored-by: Bryan White <[email protected]>
@Olshansk @bryanchriswhite
Update docusaurus/docs/develop/developer_guide/recovery_from_chain_ha…
80c3e38 
…lt.md

Co-authored-by: Bryan White <[email protected]>
@Olshansk @bryanchriswhite
Apply suggestions from code review
d6f6ced 
Co-authored-by: Bryan White <[email protected]>
@Olshansk
Fix merge conflict
a18a197
@okdas
requested changes
b7e5151
@okdas
Merge branch 'dk-upgrade-learnings' of github.com:pokt-network/poktro…
ad2b0c0 
…ll into dk-upgrade-learnings
@okdas
Merge remote-tracking branch 'origin/main' into dk-upgrade-learnings
fc679d6
@okdas
Copy link
Member Author

okdas commented Nov 27, 2024

@Olshansk I noticed you started reviewing this. I addressed @bryanchriswhite review and pushed more changes to the branch. Just FYI.

Olshansk
Olshansk approved these changes Dec 12, 2024
View reviewed changes
Copy link
Member

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed a bunch of changes. A couple lingering comments but g2g afterwards!

makefiles/localnet.mk

.PHONY: localnet_show_upgrade_plan
localnet_show_upgrade_plan: ## Shows the upgrade plan on local node
poktrolld query upgrade plan
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto regarding --home=$(POKTROLLD_HOME) --node $(POCKET_NODE) here too?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need. The upgrade happens outside tilt/k8s deployment - so these commands work as is.

makefiles/localnet.mk

.PHONY: localnet_cancel_upgrade
localnet_cancel_upgrade: ## Cancels the planed upgrade on local node
poktrolld tx authz exec tools/scripts/upgrades/authz_cancel_upgrade_tx.json --gas=auto --from=pnf
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Don't we need to --home=$(POKTROLLD_HOME) q bank balances $(ACC) --node $(POCKET_NODE)
  2. Don't we need to add fees?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No to both.
This make target is not executed against a validator running in tilt, but as a standalone validator process running locally, outside k8s deployment.

We specifically instruct turn off LocalNet before testing the upgrade.

@okdas okdas dismissed bryanchriswhite’s stale review January 24, 2025 00:24

addressed requests

@okdas okdas merged commit ea89904 into main Jan 24, 2025
12 of 13 checks passed
@okdas okdas mentioned this pull request Jan 28, 2025
Closed
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Olshansk Olshansk Olshansk approved these changes

@bryanchriswhite bryanchriswhite Awaiting requested review from bryanchriswhite

Assignees

@okdas okdas

Labels
documentation Improvements or additions to documentation push-image CI related - pushes images to ghcr.io
Projects
Shannon
Status: ✅ Done
Milestone
Beta TestNet Iteration
Development

Successfully merging this pull request may close these issues.
3 participants
@okdas @Olshansk @bryanchriswhite